fix: stop resolve() from leaking foreign envs (e.g. hatch) into the venv collection#1507
fix: stop resolve() from leaking foreign envs (e.g. hatch) into the venv collection#1507StellaHuang95 wants to merge 1 commit into
Conversation
|
From what I can tell the PR is a valid defensive fix, but the root cause is that PET has no concept of Hatch environments. Since hatch envs have pyvenv.cfg, the Venv locator here claims them. We would need to add Hatch as supported in PET. This fix is still useful as defense-in-depth (resolve shouldn't mutate state), let me look at this more to see if we still want this PR then, in addition to any PET changes. |
Hatch environments are usually regular venvs that exist in specific locations, but not always. Hatch can use any kind of environment via plugin, e.g. you could make Hatch create conda envs instead, or docker images. So I’d phrase that more generally: PET has no concept of env managers that manage a specific subset of envs of a known type – in other words, project-aware env managers like Hatch should be able to claim certain envs with a higher specificity than more generic / base env managers (which aren’t project-aware). PS: don’t you have the same issue with Poetry already? It also just creates regular venvs, right? |
|
@flying-sheep the following is merged in the PET extension which should help make sure Hatch environments do not get picked up: https://github.com/microsoft/python-environment-tools/commit/d58127285a6571fe4e0589a5c4f9220e559207…. @StellaHuang95 could you go through this PR once more now and just update it and the description given what we discussed about PET needing to recognize the env type but then about this change being a good additional stop-gate. |
@eleanorjboyd Sure, now that fix on the PET side is in, hatch envs get classified correctly at the source, so I've reframed the description of the PR to be more generic, basically |
Context
@flying-sheep reported three related bugs — #1471, #1485, #1491. The two venv-related ones both traced to the same root cause in
VenvManager.resolve().Hatch envs structurally look like venvs (they have a
pyvenv.cfg), so until very recently PET classified them asVenvand they passed throughresolveVenvPythonEnvironmentPath. Every successfulresolve()call also mutatedthis.collectionviaaddEnvironment(resolved, true). So whenever anything calledresolve()for a hatch path —Python: Select Interpreter, Pylance startup, a test runner asking for execInfo, another extension querying the env — the hatch env got persisted under venv.The original side effect dates back to commit 9363bc1 (Oct 2024), when the extension only had
systemandvenvmanagers and the author assumed any successfulresolve()was a newly discovered venv worth caching. That assumption stopped being true once the API supported third-party managers.Upstream fix has now landed in PET
As of microsoft/python-environment-tools#460 (commit d581272), PET now has a dedicated
pet-hatchlocator:LocatorKind::HatchandPythonEnvironmentKind::Hatch.Venvlocator, so it claims hatch envs first.<data_dir>/env/virtual/<project_name>/<project_id>/<venv_name>), honorsHATCH_DATA_DIR, and reads[tool.hatch.dirs.env].virtualfrompyproject.toml/hatch.toml.That fixes the structural source of the hatch-as-venv misclassification.
What this PR does
This PR is a complementary, defense-in-depth correction in
src/managers/builtin/venvManager.ts:if (resolved) { if (resolved.envId.managerId === `${PYTHON_EXTENSION_ID}:venv`) { - // This is just like finding a new environment or creating a new one. - // Add it to collection, and trigger the added event. - this.addEnvironment(resolved, true); - // We should only return the resolved env if it is a venv. return resolved; } } return undefined;resolve()is now a pure query: "given a path, are you the manager for it?" It still returns the resolved env (every consumer ofresolve()keeps working), it just stops mutating state. Cached venvs continue to land inthis.collectionthrough the normal discovery and creation paths, which is the only path that should be writing to it.Why keep this PR alongside the PET fix
Even with PET correctly classifying hatch envs,
VenvManager.resolve()mutatingthis.collectionis an invariant violation worth correcting:pyvenv.cfg-shaped env (tox, nox, custom bootstrap scripts, future tools) would re-introduce the same class of leak. Withresolve()as a pure query, no future overlap can silently grow the venv collection.